Novel HPC Technologies for Scalable CAE: The Case for Parallel I/O and File Systems

نویسنده

  • Stan Posey
چکیده

As HPC continues its aggressive platform migration from proprietary supercomputers and Unix servers to HPC clusters, expectations grow for clusters to meet the I/O demands of increasing fidelity in CAE modeling and data management in the CAE workflow. Cluster deployments have increased as organizations seek ways to costeffectively grow compute resources for CAE applications, and during this migration many also implemented conventional network attached storage (NAS) architectures to simplify IT administration and further reduce costs. While legacy NAS implementations offer several advantages of shared file systems, most are too limited in scalability for effective management of I/O demands with parallel CAE applications. As such, a new storage migration is underway to replace legacy (serial) NAS with parallel NAS architectures and parallel file systems. This new class of parallel file system and shared storage technology was developed to scale I/O in order to extend the overall scalability of CAE simulations on clusters. This paper examines CAE motivation for shared parallel file systems and storage, for requirements of multi-physics LS-DYNA applications on conventional clusters with proper balance for I/O. Model parameters such as size, element types, schemes of implicit and explicit (and coupled), and a variety of simulation conditions can produce a wide range of computational behavior and I/O data management demands. The benefits of a Panasas storage implementation are introduced for such broad requirements, through examples of CAE workflows for a variety of production-level applications in industry.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Tuning HDF5 for Lustre File Systems

HDF5 is a cross-platform parallel I/O library that is used by a wide variety of HPC applications for the flexibility of its hierarchical object-database representation of scientific data. We describe our recent work to optimize the performance of the HDF5 and MPI-IO libraries for the Lustre parallel file system. We selected three different HPC applications to represent the diverse range of I/O ...

متن کامل

Scaling Security for Big, Parallel File Systems

The need for petaand exabyte scale parallel file systems that support high-performance computing (HPC) has been rapidly increasing. These systems have unique demands, different from those of traditional distributed file systems. As a result, securing I/O in big, parallel file systems without significantly impacting performance has proven challenging. Parallel file systems are commonly composed ...

متن کامل

pNFS and Linux: Working Towards a Heterogeneous Future

Heterogeneous and scalable remote data access is a critical enabling feature of widely distributed collaborations. Parallel file systems feature impressive throughput, but sacrifice heterogeneous access, seamless integration, security, and cross-site performance. Remote data access tools such as NFS and GridFTP provide secure access to parallel file systems, but either lack scalability (NFS) or...

متن کامل

vPFS: Bandwidth Virtualization of Parallel Storage Systems

This paper presents vPFS, a new parallel file system performance management approach to support the allocation of shared storage bandwidth on a per-application basis. Existing parallel file systems are unable to differentiate I/O requests from different applications and meet per-application bandwidth requirements. This limitation presents an increasing hurdle for applications to achieve their d...

متن کامل

The Edge Node File System: A Distributed File System for High Performance Computing

The concept of using Internet edge nodes for High Performance Computing (HPC) applications has gained acceptance in recent times. Many of these HPC applications also have large I/O requirements. Consequently, an edge node file system that efficiently manages the large number of files involved can assist in improving application performance significantly. In this paper, we discuss the design of ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2008